2 research outputs found
A geometrically constrained multimodal approach for convolutive blind source separation
A novel constrained multimodal approach for convolutive blind source separation is presented which incorporates video information related to geometrical position of both the speakers and the microphones, and the directionality of the speakers into the separation algorithm. The separation is performed in the frequency domain and the constraints are incorporated through a penalty function-based formulation. The separation results show a considerable improvement over traditional frequency domain convolutive BSS systems such as that developed by Parra and Spence. Importantly, the inherent permutation problem in the frequency domain BSS is potentially solve
Virtual friend: tracking and generating natural interactive behaviours in real video
The aim of our research is to create a “virtual
friend” i.e., a virtual character capable of responding
to actions obtained from observing a real person in
video in a realistic and sensible manner. In this paper,
we present a novel approach for generating a variety
of complex behavioural responses for a fully articulated
“virtual friend” in three dimensional (3D) space.
Our approach is model-based. First of all, we train a
collection of dual Hidden Markov Models (HMMs) on
3D motion capture (MoCap) data representing a number
of interactions between two people. Secondly, we
track 3D articulated motion of a single person in
ordinary 2D video. Finally, using the dual HMM, we
generate a moving “virtual friend” reacting to the
motion of the tracked person and place it in the
original video footage. In this paper, we describe our
approach in depth as well as present the results of
experiments, which show that the produced behaviours
are very close to those of real people